1. Field of the Invention
The present invention generally relates to the field of computer-based testing, and in particular, the present invention relates to customizing a visual presentation of a computer-based test using templates that define a presentation format of the computer-based test and plugins that contain data content of the computer-based test.
2. Background of the Related Art
For many years, standardized testing has been a common method of assessing examinees as regards educational placement, skill evaluation, etc. Due to the prevalence and mass distribution of standardized tests, computer-based testing has emerged as a superior method for providing standardized tests, guaranteeing accurate scoring, and ensuring prompt return of test results to examinees.
Tests are developed based on the requirements and particulars of test developers. Typically, test developers employ psychometricians or statisticians and psychologists to determine the specific requirements specific to human assessment. These experts often have their own, unique ideas regarding how a test should be presented and regarding the necessary contents of that test, including the visual format of the test as well as the data content of the test. Therefore, a particular computer-based test has to be customized to fulfill the client's requirements.
FIG. 1 illustrates a prior art process for computerized test customization, denoted generally by reference numeral 10. First, a client details the desired test requirements and specifications, step 12. The computerized test publisher then creates the tools that allow the test publisher to author the items, presentations, etc., required to fulfill the requirements, step 14. The test publisher then writes an item viewer, which allows the test publisher to preview what is being authored, step 16.
An item presenter is then written to present the new item, for example, to the test driver, step 18. Presenting the new item to the test driver requires a modification of the test driver's executable code. The test driver must be modified so that it is aware of the new item and can communicate with the new item presenter, step 20. The test packager must then also be modified, step 22. The test packager, which may also be a compiler, takes what the test publisher has created and writes the result as new object codes for the new syntax. Subsequently, the scoring engine must also be modified to be able to score the new item type, step 24. Finally, the results processor must be modified to be able to accept the new results from the new item, step 26. This process requires no less than seven software creations or modifications to existing software.
Previous test drivers had a fixed look and feel to the visual presentation of tests. For example, if a particular test driver was programmed to present navigation buttons on the bottom of the test screen or to place the title bar at the top of the screen, every test delivered by that test driver would have the identical visual presentation. However, as stated previously, a client might wish to modify or customize the visual presentation of a test. A client might desire customization for psychometric reasons; for example, the client knows that examinees function better with exams presented in a particular way. Or, a client might want a particular visual presentation for a test for marketing reasons. In previous computer-based tested systems, the test driver would have to undergo massive reprogramming, as described above, in order to accommodate a particular client's wishes.
U.S. Pat. No. 5,827,070 (Kershaw et al.) and U.S. Pat. No. 5,565,316 (Kershaw et al.) are incorporated herein by reference. The '070 and '316 patents, which have similar specifications, disclose a computer-based testing system comprising a test development system and a test delivery system. The test development system comprises a test document creation system for specifying the test contents, an item preparation system for computerizing each of the items in the test, a test preparation system for preparing a computerized test, and a test packaging system for combining all of the items and test components into a computerized test package. The computerized test package is then delivered to authorized examinees on a workstation by the test delivery system.
FIGS. 2A and 2B illustrate the screen format during a delivery unit and the testing tools utilized in the control area, respectively, as disclosed in the '070 and '316 patents. The screen format during a delivery unit is preferably divided into three main sections; a title line 30, an item presentation area 32, and a primary control area 34 as illustrated in FIG. 2A. The title line 30 is preferably presented as one solid gray bar at the top of the screen. It should be understood that numerous other color combinations are possible. The title line 30 is capable of displaying various information comprising the test or taking the test. For instance, the title line 30 may include the time remaining in the test or test section. Preferably, time is displayed automatically although an examinee may optionally turn it off. In a preferred embodiment, remaining time is displayed left justified on the title line 30 in HH:MM format until the last few minutes of the test. At that point, the display format changes to MM:SS and flashes for three seconds so that the examinee is alerted that the time remaining for taking the test is nearly over. Other information in the title line 30 may include the name of the computerized test and program-specified text pertinent to what is being presented (e.g., section name). Information to help orient the examinee is also preferably displayed in title line 30. For instance, when an item or tutorial screen is displayed in the presentation area 32, the notation “xx of yy” or “xx” appears in title line 30. The “xx” refers to the item number within the test or section, or the screen within the tutorial. The “yy” indicates the total number of items in the test or section, or screens within the tutorial. Additional orientation information to be provided to the examinee in the title line 30 may include the descriptive word such as “HELP” “REVIEW” and “DIRECTION” to indicate a currently displayed screen.
The presentation area 32 of the screen is used to display items screen such as the text and graphics and non-item screens such as direction screens, message screens, HELP screens and REVIEW screens. Direction screens are used to display directions for the test, section, and others. Message screens display information and examinee options at transition points during the test session to control the flow of the test. Transition points indicate where a new section or new item is to be displayed or when the test delivery application moves from an item screen to a non-item screen. HELP screens and REVIEW screens respectively enable an examinee to interact with the HELP and REVIEW facilities.
The primary control area 34 preferably provides testing tools for giving the examinee a degree of control over the testing session. In a preferred embodiment, there are ten testing tools (also referred to as “primary controls”). Each tool has its own icon. Icons are pictorial representation of a function available to a user, which can be activated by selecting that icon. Referring to FIG. 2B, the NEXT icon 54, the PREV icon 52, the REVIEW icon 42 and HELP icon 50 can be used by the examinee to move from one screen to another screen. When the NEXT icon 54 is selected, the examinee can move on to the next screen. Selecting the PREV icon 52 enables the examinee to move back to the previous screen. The HELP icon 50 can be selected by the examinee to invoke the HELP facility. When HELP is invoked, the examinee moves to a HELP screen to retrieve previously presented direction and information about topics covered in the tutorials. The examinee is returned to the screen from which HELP was invoked when the Help screen is exited. The MARK icon 44 enables the examinee to mark an item for review. In a preferred embodiment, both answered and unanswered items can be marked. A marked item is indicated on an item screen by displaying a checkmark in the MARK icon 44. The checkmark may also appear next to the marked item in the Review screen when the REVIEW facility is invoked after an item has been marked. The examinee can unmark an item by clicking on the MARK icon 44 a second time. However, preferably the examinee need not unmark items in order to leave a section. The REVIEW icon 42, when selected, presents the review screen to the examinee listing the items in the section in the order they were presented to the examinee, along with any group or set paraphrase associated with item and an indication of whether the item has been marked by the examinee from the item screen. The examinee then has the ability to go directly to any item in a test section by clicking, as described above. Preferably, the examinee can invoke the REVIEW facility from any item screen, and the REVIEW screen will display the status of all the items in the section regardless of whether all of the items had been presented. In a further preferred embodiment, the examinee may skip some items by advancing to a subsequent item. The ERASE icon 46 enables the examinee to reset all selected choices for the current item to their original state. The TIME icon 40 allows the examinee to turn on and off the remaining time display in the title line 30. The EXIT icon 38 allows the examinee to leave the current section of the test. The QUIT icon 36 allows the examinee to quit the test. The CALC icon 48 allows the examinee to use on-screen calculator.
A testing tool is said to exist if it appears on every screen. The existence of each of the depicted icons; PREV, CALC, QUIT, EXIT, TIME, REVIEW, MARK and ERASE is specified by each test script or the section configuration file. The NEXT and HELP tools preferably exist in all tests. Testing tools that do not exist should not appear on the screen, and in a preferred embodiment, the location of the remaining tools is adjusted to close any gaps left by non-existent tools. Test scripts can define the existence of tools to limit the ways in which examinees can navigate through the test. For example, a program can define a forward-progression-only test by eliminating the PREV and REVIEW tools. A testing tool is said to be available if it exists and can be used. Preferably, a testing tool is displayed in black if available and in gray when it is not. For instance, in preferred embodiments, the NEXT icon 54, the PREV icon 52, ERASE icon 46, MARK icon 44, and the CALC icon 48 are available only from item screens. However, the HELP icon 50 is available from all screens. The REVIEW icon 42 is available from item screens and group or set directions screens while the TIME icon 40 may or may not be available from directions screens depending upon whether the section configuration file indicates timing is to start before or after the presentation of directions.
U.S. Pat. No. 5,513,994 (Kershaw et al.), which is incorporated herein by reference, discloses a centralized administrative system and method of administering standardized test to a plurality of examinees. The administrative system is implemented on a central administration workstation and at least one test workstation located in different rooms at a test center. The administrative system software, which provides substantially administrative functions, is executed from the central administration workstation. The administrative system software, which provides function carried out in connection with a test session, is executed from the testing workstations.
None of the Kershaw et al. patents appear to make any mention of how modifications may be made to the computer-based testing system to incorporate a particular client's test specification regarding the visual presentation of the test. The '070 and '316 patents only disclose a screen format that is preferably divided into three main sections: a title line, an item presentation area, and a primary control area, where the control area contains a specific set of non-customizable tools. The patents make no mention of being able to customize, for example, the title line or the primary control area, to allow different versions of these sections to be presented to an examinee during a single test.
What is required is a system that allows the test publisher or publisher to have full control over the visual presentation of the content, navigation controls, titles, and other ancillary display information that are delivered to an examinee during a test.